chore: Update sqlparser to 0.63.0 - #25278
Merged
Merged
Conversation
Pin sqlparser to the head of apache/datafusion-sqlparser-rs (0821cbdd5cbdc27f9c7205466d357deef632c936) and update DataFusion for the AST changes, to validate the upcoming 0.63.0 release. See apache/datafusion-sqlparser-rs#2453
sqlparser now lists the [NOT] JSON alternatives in the expected-token error message after IS
Pin to the head of the release PR (apache/datafusion-sqlparser-rs#2481), which includes the version bump to 0.63.0 plus two AST changes merged since the previous pin: - LIKE / ILIKE / SIMILAR TO escape_char is now Option<Box<Expr>>; DataFusion still only accepts a single-character string literal - New MergeAction::DoNothing variant, rejected with a not implemented error
Picks up sqlparser_derive 0.6.0 and the source-order function clause visitor change (apache/datafusion-sqlparser-rs#2477)
# Conflicts: # datafusion/sql/src/unparser/expr.rs
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25278 +/- ##
==========================================
- Coverage 82.33% 82.32% -0.01%
==========================================
Files 1137 1137
Lines 431887 431933 +46
Branches 431887 431933 +46
==========================================
+ Hits 355579 355605 +26
- Misses 54814 54831 +17
- Partials 21494 21497 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Resolve conflict in datafusion/sql/src/query.rs: keep main's rewritten ORDER BY ALL handling and clone `OrderByOptions`, which is no longer `Copy` in sqlparser 0.63. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alamb
marked this pull request as ready for review
September 15, 2026 20:36
Contributor
Author
|
run benchmark sql_planner |
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing update-sqlparser-0.63 (50f7984) to a0631ed (merge-base) diff Run configurationrun benchmark sql_plannerResults will be posted here when complete File an issue against this benchmark runner |
comphead
reviewed
Sep 15, 2026
| statement error This feature is not implemented: Calling array_agg: LIMIT not supported in function arguments: 1 | ||
| SELECT array_agg(c13 LIMIT 1) FROM aggregate_test_100 | ||
|
|
||
| statement error This feature is not implemented: Calling array_agg: WHERE not supported in function arguments: c2 > 1 |
|
🤖 Benchmark completed (GKE) | trigger Instance: Comparing update-sqlparser-0.63 (50f7984) to a0631ed (merge-base) diff Run configurationrun benchmark sql_plannerCPU Details (lscpu)Details
Resource Usagesql_planner — base (merge-base)
sql_planner — branch
File an issue against this benchmark runner |
comphead
reviewed
Sep 15, 2026
| } | ||
| ast::MergeAction::Delete { .. } => MergeIntoAction::Delete, | ||
| ast::MergeAction::DoNothing { .. } => { | ||
| return not_impl_err!("MERGE DO NOTHING is not supported"); |
Contributor
There was a problem hiding this comment.
this err message is super confusing IMO
Contributor
Author
There was a problem hiding this comment.
I think the DO NOTHING clause is confusing in general -- I think this comment explains it: https://github.com/apache/datafusion/pull/25278/changes#r4021127166
Contributor
Author
There was a problem hiding this comment.
Clarified the comment in 7853731a33
comphead
reviewed
Sep 15, 2026
| when not matched then insert *; | ||
|
|
||
| statement error DataFusion error: This feature is not implemented: MERGE DO NOTHING is not supported | ||
| merge into target using source on target.id = source.id |
Contributor
There was a problem hiding this comment.
its useless statement, wondering why it is considering to be a valid sql
Contributor
Author
There was a problem hiding this comment.
Do you mean why is DO NOTHING part of SQL?
I think it is supported by POSTRGES, see
https://www.postgresql.org/docs/current/sql-merge.html
Which says
When DO NOTHING is specified, the source row is skipped. Since actions are evaluated in their specified order, DO NOTHING can be handy to skip non-interesting source rows before more fine-grained handling.
🤷
Contributor
There was a problem hiding this comment.
oh this is a real thing 🤦
alamb
enabled auto-merge
September 16, 2026 18:53
Contributor
Author
|
Thank you for the review @comphead -- I think I addressed all the comments. Let me know if there is something else that I missed |
alamb
disabled auto-merge
September 16, 2026 18:55
Contributor
Author
|
run benchmark sql_planner |
alamb
enabled auto-merge
September 16, 2026 18:55
|
🤖 Benchmark running (GKE) | trigger CPU Details (lscpu)Comparing update-sqlparser-0.63 (7853731) to 140c7c5 (merge-base) diff Run configurationrun benchmark sql_plannerResults will be posted here when complete File an issue against this benchmark runner |
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Sep 16, 2026
Contributor
Author
|
🤔 the sqlite etended tests failed:
|
|
🤖 Benchmark completed (GKE) | trigger Instance: Comparing update-sqlparser-0.63 (7853731) to 140c7c5 (merge-base) diff Run configurationrun benchmark sql_plannerCPU Details (lscpu)Details
Resource Usagesql_planner — base (merge-base)
sql_planner — branch
File an issue against this benchmark runner |
Contributor
Author
|
Benchmarks seem to show no obvious differences in my anaylsis |
github-actions
Bot
added
documentation
Contributor
Author
|
To merge this PR I need to update the contents of datafusion-testing to update expected errors: |
alamb
added a commit
to apache/datafusion-testing
that referenced
this pull request
Sep 17, 2026
DataFusion is upgrading to sqlparser 0.63.0, which adds recursion guards to more parse functions so every expression consumes one more level of the parser recursion budget. To keep previously-parseable queries working, DataFusion is raising the default `datafusion.sql_parser.recursion_limit` from 50 to 51 (apache/datafusion#25278). Update the 275 `RecursionLimitExceeded` expectations across 15 sqlite test files to expect `(current limit: 51)`. The queries still fail with the new limit, only the limit reported in the error message changes. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Author
Thanks to @viirya I merged the test updates and will try to merge this one agian |
alamb
enabled auto-merge
September 17, 2026 16:16
viirya
approved these changes
Sep 17, 2026
Contributor
Author
|
WoohoO! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
0.63.0around 2026-08-31 datafusion-sqlparser-rs#2453Rationale for this change
sqlparser0.63.0 has been released (changelog), so update DataFusion to use it.This PR is based on the testing PR #24926
What changes are included in this PR?
sqlparserto 0.63.0Are these changes tested?
Yes, by existing CI (including sqllogictests).
Are there any user-facing changes?
Updated
sqlparserversion. Error messages for a few unsupported syntax cases have changed slightly (see updated test indatafusion/sql/tests/cases/params.rs).All existing tests continue to pass